Multiple Hypothesis Testing in Conjoint Analysis

Abstract Conjoint analysis is widely used for estimating the effects of a large number treatments on multidimensional decision-making. However, it this substantive advantage that leads to statistically undesirable property, multiple hypothesis testing. Existing applications conjoint except few do not correct hypotheses be tested, and empirical guidance choice testing correction methods has been provided. This paper first shows even when none any effect, standard pipeline produces at least one significant estimate average marginal component in more than 90% experimental trials. Then, we conduct simulation study compare three well-known correction, Bonferroni Benjamini–Hochberg procedure, adaptive shrinkage (Ash). All are accurate recovering truth conventional without correction. Moreover, Ash method outperforms avoiding false negatives, while reducing positives similarly other methods. Finally, show how conclusions drawn from may differ with by reanalyzing public attitudes toward immigration partner countries trade agreements.